An axiomatized PDE model of deep neural networks

Inspired by the relation between deep neural network (DNN) and partial differential equations (PDEs), we study the general form of the PDE models of deep neural networks. To achieve this goal, we formulate DNN as an evolution operator from a simple base model. Based on several reasonable assumptions, we prove that the evolution operator is actually determined by convection-diffusion equation. This convection-diffusion equation model gives mathematical explanation for several effective networks. Moreover, we show that the convection-diffusion model improves the robustness and reduces the Rademacher complexity. Based on the convection-diffusion equation, we design a new training method for ResNets. Experiments validate the performance of the proposed method

Diffusion Mechanism in Residual Neural Network: Theory and Applications

Diffusion, a fundamental internal mechanism emerging in many physical processes, describes the interaction among different objects. In many learning tasks with limited training samples, the diffusion connects the labeled and unlabeled data points and is a critical component for achieving high classification accuracy. Many existing deep learning approaches directly impose the fusion loss when training neural networks. In this work, inspired by the convection-diffusion ordinary differential equations (ODEs), we propose a novel diffusion residual network (Diff-ResNet), internally introduces diffusion into the architectures of neural networks. Under the structured data assumption, it is proved that the proposed diffusion block can increase the distance-diameter ratio that improves the separability of inter-class points and reduces the distance among local intra-class points. Moreover, this property can be easily adopted by the residual networks for constructing the separable hyperplanes. Extensive experiments of synthetic binary classification, semi-supervised graph node classification and few-shot image classification in various datasets validate the effectiveness of the proposed method

Bone Morphogenetic Protein-2 Promotes Osteosarcoma Growth by Promoting Epithelial-Mesenchymal Transition (EMT) Through the Wnt/b-Catenin Signaling Pathway

The correlation between BMP-2 and osteosarcoma growth has gained increased interest in the recent years, however, there is still no consensus. In this study, we tested the effects of BMP-2 on osteosarcoma cells through both in vitro and in vivo experiments. The effect of BMP-2 on the proliferation, migration and invasion of osteosarcoma cells was tested in vitro. Subcutaneous and intratibial tumor models were used for the in vivo experiments in nude mice. The effects of BMP-2 on EMT of osteosarcoma cells and the Wnt/β-catenin signaling pathway were also tested using a variety of biochemical methods. In vitro tests did not show a significant effect of BMP-2 on tumor cell proliferation. However, BMP-2 increased the mobility of tumor cells and the invasion assay demonstrated that BMP-2 promoted invasion of osteosarcoma cells in vitro. In vivo animal study showed that BMP-2 dramatically enhanced tumor growth. We also found that BMP-2 induced EMT of osteosarcoma cells. The expression levels of Axin2 and Dkk-1 were both down regulated by BMP-2 treatment, while β-catenin, c-myc and Cyclin-D1 were all upregulated. The expression of Wnt3α and p-GSK-3β were also significantly upregulated indicating that the Wnt/β-catenin signaling pathway was activated during the EMT of osteosarcoma driven by BMP-2. From this study, we can conclude that BMP-2 significantly promotes growth of osteosarcoma cells (143B, MG63), and enhances mobility and invasiveness of tumor cells as demonstrated in vitro. The underlying mechanism might be that BMP-2 promotes EMT of osteosarcoma through the Wnt/β-catenin signaling pathway. © 2019 Orthopaedic Research Society. Published by Wiley Periodicals, Inc. J Orthop Res 37:1638–1648, 2019. © 2019 Orthopaedic Research Society. Published by Wiley Periodicals, Inc

Real GDP in pre-war East Asia: a 1934–36 benchmark purchasing power party comparison with the U.S.

This article provides estimates of purchasing power parity (PPP) converters for expenditure side GDP of Japan/China, Japan/U.S. and China/U.S. in 1934-36 through a detailed matching of prices for more than 50 types of goods and services in private consumption and about 20 items or sectors for investment and government expenditure. Linking with the earlier studies on the price levels of Taiwan and Korea relative to Japan, we derive the mid-1930s benchmark PPP adjusted per capita income of Japan, China, Taiwan and Korea at 32, 11, 23, and 12 percent of the U.S. level respectively. These estimates correct the consistent downward bias in East Asian income levels based on market exchange rate conversions. Compared with Angus Maddison's estimates based on the 1990 benchmark back-projection, our current-price based result are 18 and 44 percent lower for Japan and Korea, and 4 and 10 percent higher for Taiwan and China respectively in the mid-1930s. We develop a preliminary theoretical and empirical framework to examine the possible source of the biases in the back-projection method. The article ends with a discussion on historical implications of our findings on the initial conditions and long-term growth dynamics in East Asia. Copyright � 2007 The Authors; Journal compilation � International Association for Research in Income and Wealth 2007.